1. Field of the Invention
The present invention relates to an image pickup apparatus for picking up an object image.
2. Related Background Art
Conventionally, in forming a color image, an image pickup technique is widely used which picks up an object image by a single image pickup element in which pixels each provided with any one of primary color filters of red (R), green (G) and blue (B) are arranged in mosaic and generates luminance information and color information corresponding to the number of pixels by subsequent signal processing. In many cases, a color filter array of the image pickup element used in this technique is the Bayer array. This image pickup technique can realize significant cost reduction without degrading quality of an image so much compared with an image pickup method of a three plate system that separates an object image into R, G and B in advance using a beam splitter for color separation and picking up an image by image pickup elements prepared for respective colors.
However, the above-mentioned conventional technique has the following problems. In general, image pickup for obtaining good image characteristics is composed of a first process for forming an object image by an optical device, a second process for adjusting a high frequency component of a spatial frequency characteristic of the object image to be suppressed, a third process for photoelectrically converting the object image with the spatial frequency characteristic adjusted, and a fourth process for correcting a response to an obtained electric signal according to a spatial frequency. In this image pickup, since sampling of an optical image is performed by an image pickup element with a limited number of pixels, it is necessary to reduce components equal to or greater than the Nyquist frequency peculiar to the image pickup element in spatial frequency characteristics of the optical image. Here, the Nyquist frequency means a frequency that is a half of a sampling frequency depending on a pixel pitch. Therefore, the optimized series of processes adjusts an optical image subjected to sampling to be an optical image of characteristics corresponding to the Nyquist frequency peculiar to the image pickup element, thereby obtaining a high quality image in which aliasing distortion is not conspicuous, that is, moiré is not conspicuous.
A Modulation Transfer Function (MTF) that is a spatial frequency transmission characteristic of an image is an evaluation amount with which a characteristic concerning sharpness of a digital still camera, a video camera or the like can be represented well. Specific elements affecting this MTF are a focusing optical system functioning as an optical device, an optical lowpass filter for limiting a band of an object image, an opening shape of an photoelectrical conversion area of an image pickup element, digital aperture correction and the like. An overall MTF representing final image characteristics is given as a product of MTF of each element. That is, it is sufficient to find MTFs for the above-mentioned first to fourth processes, respectively, and calculate a product of the MTFs. However, since digital filtering that is the fourth process is applied to an image output that has already been subjected to sampling by the image pickup element, it is unnecessary to take into account a high frequency wave exceeding the Nyquist frequency.
Therefore, a configuration for reducing components equal to or greater than the Nyquist frequency peculiar to an image pickup element in spatial frequency characteristics of an optical image means a configuration in which components equal to or greater than the Nyquist frequency are few in a product of the MTF of the first process, the MTF of the second process and the MTF of the third process excluding the fourth process. Here, in the case in which viewing of a still image is considered a premise as in a digital still camera, it is necessary to take into account the fact that an image giving a good feeling of resolution can be easily realized if response in a frequency slightly lower than the Nyquist frequency is higher even if a high frequency wave exceeding the Nyquist frequency is not zero but remains more or less.
In the formation of an object image by the focusing optical system that is the first process, in general, it is easier to correct optical aberration in a center of a screen than in a periphery of the screen. When it is attempted to obtain a good image in the periphery of the screen, it is necessary to obtain extremely good characteristics close to a diffraction limit MTF that depends on an F-number of a focusing lens in the center of the screen. In recent years, this necessity has been increasing as an image pickup element uses smaller pixels. Therefore, it is better to consider an MTF on the assumption that the focusing optical system is an ideal aplanatic lens.
In addition, in an image pickup element in which light receiving openings of a width d are laid without of the light receiving opening is the same as a pixel pitch, a response value of the third process at the Nyquist frequency u=½d is rather high. Due to this response, it is a general practice to trap the vicinity of the Nyquist frequency in the second process in order to lower the overall MTF in the vicinity of the Nyquist frequency.
In the second process, an optical lowpass filter is usually used. A material having a birefringence characteristic such as that of quartz is used for the optical lowpass filter. In addition, a diffraction grating of a phase type as disclosed in JP 2000-066141 A may be used. When a birefringent plate is intervened in an optical path of an optical device and the optical axis is slanted to be arranged in parallel with a horizontal direction of a focusing surface, an object image by a normal ray and an object image by an abnormal ray are formed deviating by a predetermined amount in the horizontal direction. Trapping a specific spatial frequency by the birefringent plate means deviating a bright part and a dark part of a stripe of the spatial frequency so as to overlap each other. An MTF by the optical lowpass filter is represented by Expression (1):R2(u)=|cos (π·u·ω)|  (1)where Rs(u) is response, u is a spatial frequency of an optical image and ω is a separation width of an object image.
If a thickness of the birefringent plate is selected appropriately, it is possible to reduce the response to zero in the Nyquist frequency of the image pickup element. If the diffraction grating is used, it is possible to realize the same effect by separating an optical image into a plurality of images of a predetermined positional relationship and superimposing the images by diffraction. However, it is necessary to grow crystal such as quartz or lithium niobate and then grind it to be thin in order to manufacture the birefringent plate. Thus, the birefringent plate becomes very expensive. In addition, the diffraction grating is also expensive because a highly precise fine structure is required.
On the other hand, JP 2001-078213 A discloses a technique that makes an effective light receiving opening larger than a pixel pitch to suppress an MTF of pixels equal to or greater than the Nyquist frequency by using a compound eye lens although an image pickup system of a single plate type is used. However, since image shift depending on an object distance occurs due to the compound eye, a sampling pitch of an object image becomes unequal in a distance other than a reference object distance. That is, misregistration occurs. Therefore, a predetermined image performance is not always realized regardless of object conditions.
Moreover, as disclosed in JP 01-014749 B (FIG. 5), it is also attempted to suppress response to a high spatial frequency by forming a photoelectrical conversion area of pixels in an intricate shape with respect to neighboring pixels. However, since a shape of the pixels becomes complicated, an extremely fine structure is required. In addition, since each pixel divides a plane, an effect is not easily realized if an object image projected on the pixels has, for example, a slant line along the dividing line. In addition, in each pixel of a color image pickup element, only light transmitted through a predetermined optical filter among an incident light flux is photoelectrically converted, and the light is outputted as an electric signal. Thus, light that could not have been transmitted through the optical filter is disposed of as heat or the like.
FIG. 26 shows an array of photoelectrical conversion areas 901 on an image pickup element. As shown in FIG. 27, a microlens 902 for magnifying area of an opening is provided for each of the photoelectrical conversion areas 901. FIG. 28 is a perspective view of the microlens 902. As shown in the figure, the microlens 902 is a lens having a positive power and has a function of converging received light fluxes to the photoelectrical conversion areas 901 of the image pickup element.
For example, in a CCD image pickup element having pixels with primary color filters arranged in mosaic, which is said to have good color reproducibility, optical filters of red (R), green (G) and blue (B) are arranged one by one between the microlens 902 and the photoelectrical conversion area 901. In this case, in the pixels having the optical filters of R arranged, only red light is photoelectrically converted, and blue light and green light are absorbed by the optical filter and generate heat. In the pixels having the optical filters of G arranged, blue light and red light are not photoelectrically converted but outputted as heat in the same manner. In the pixels having the optical filters of B arranged, green light and red light are not photoelectrically converted but outputted as heat in the same manner. FIG. 25 shows a spectral transmissivity characteristic of the color filters of R, G and B in an image pickup element. Since infrared ray has high transmissivity, infrared ray cut filters for cutting a wavelength of 650 nm or more are additionally used in stack. As is seen from this, only ⅓ of visible light is effectively used in one pixel.
Considering a utilization efficiency for each color of R, G and B more in detail, for example, a ratio of area of R, G and B pixel of a color image pickup element of the Bayer array shown in FIG. 29 is ¼: 2/4:¼ when area of a unit constituting a regular array is assumed to be one. Thus, a utilization ratio of green light when an overall amount of light is one is ⅓× 2/4=⅙ as a product of a term of wavelength selectivity and a term of an area ratio, those for red light and blue light are ⅓×¼= 1/12, respectively. When these are totaled, ⅙+ 1/12+ 1/12=⅓. Therefore, the utilization efficiency is still ⅓. To the contrary, when an overall amount of light is assumed to be one, ⅔× 2/4=⅓ of green light and ⅔×¼=⅙ of red light and blue light are not effectively utilized among the overall amount of light.
The above description is for the image pickup element using primary color filters. However, ⅓ of visible light is not photoelectrically converted and is not effectively utilized in an image pickup element using complementary color filters. In this way, the conventional image pickup element of the single plate type using either primary color filters or complementary color filters has a bad light utilization efficiency because an image pickup surface is divided by color filters.
On the other hand, JP 08-182006 A discloses a structure of an image pickup element that eliminates such waste of an amount of light. In this image pickup element, a prism is arranged for each spatial pixel, and object light whose color is separated by the prism is received by three color pixels of R, G and B. However, the color pixels have a size approximately ⅓ of the spatial pixels. If it is attempted to make a sensor of a small pixel pitch, an extremely fine structure is required of the color pixels, but there is a limit in making the pixel pitch smaller.